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The present study examines the effectiveness of incorporating worked examples with prompts for self- 
explanation into a middle school math textbook. Algebra 1 students (N=75) completed an equation- 
solving unit with reform textbooks either containing the original practice problems or in which a 
portion of those problems were converted into correct, incorrect, or incomplete examples. Students 
completed pre- and posttest measures of conceptual understanding, procedural problem-solving 
skill, and error anticipation. Results indicate the example-based textbook assignments increased 
students’ procedural knowledge and their ability to anticipate errors one might make when solving 
problems. Differences in students’ anticipation of various types of errors are also examined. 
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Introduction 

Over the past few decades, a significant body of work in cognitive science has provided evidence 
for the Worked Example principle, which argues for having learners spend some of their practice 
time studying worked out examples of problem solutions rather than solving all of the problems 
themselves (e.g., Sweller & Cooper, 1985; Sweller, 2006). Several studies have confirmed the 
effectiveness of studying worked examples in computer-based classrooms (Booth, Lange, Koedinger, 
& Newton, 2013; Kim, Weitz, Heffernan, & Krach, 2009; Paas, 1992) and in traditional mathematics 
classrooms (Booth, Cooper, Donovan, Huyghe, Koedinger & Pare-Blagoev, 2015; Booth, Oyer, 
Pare-Blagoev, Elliot, Barbieri, & Koedinger, 2015), though none to date have examined its effect in 
reform classrooms. Further, studies have also expanded on the worked example principle to suggest 
that having students self-explain the examples (Renkl, Stark, Gruber, & Mandl, 1998), gradually 
fading the support provided in correct examples (Atkinson, Renkl, & Merrill, 2003), and having 
students also consider incorrect examples (Siegler, 2002) can have enhanced learning benefits for 
conceptual and procedural knowledge, both of which are critical in mathematics (NMAP, 2008). 

Despite this solid body of support, these practices are not typically undertaken in real world 
classrooms; the incorporation of incorrect examples is looked upon as a particularly foreign practice. 
In the United States, teachers tend to shy away from talking about errors (Lannin, Townsend, & 
Barker, 2006) at least in part due to the fear that their students will adopt the errors in their own 
problem solving (Santagata, 2004). However, students can learn a great deal from considering errors. 
Two established theories provide explanations for how studying errors can be useful for learning. 
Ohlsson’s (1996) theory suggests that explaining why an error is wrong can help learners identify the 
particular features of the problem that make the solution incorrect; this can lead to refinement of 
problem solving skills and remediation of misconceptions. In addition, Siegler’s (1996) overlapping 
waves theory maintains that individuals know and use a variety of (correct and incorrect) strategies 
for solving problems, and those strategies compete for use each time a problem is encountered. 
Studying errors is thought to be an effective way of helping learners accept that those particular 
strategies are wrong and prompting them to construct and strengthen other, correct strategies 
(Siegler, 2002). 

Consistent with these theories, recent research has found that error reflection is indeed beneficial 
to learning. For example, having students think about and correct their own errors can lead to greater 
engagement and improved problem solving skill (Cherepinsky, 2011; Henderson & Harper, 2009); 
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studying the errors made by others may be even more effective (Yerushalmi & Polingher, 2006), in 
part because it exposes students to multiple perspectives other than their own (Siegler & Chen, 
2008). Incorporating incorrect examples into class assignments and prompting students to explain 
why they are incorrect has been found to be particularly beneficial for improving conceptual 
understanding (Booth et al., 2013) 

Thus, there is growing consensus that students can learn effectively from explaining errors. 
However, textbooks don’t often include incorrect examples, and creating materials and lessons that 
include incorrect examples can be very time consuming for teachers. Some curriculum materials 
have recently been developed and tested and are available for use (e.g., see Booth et al., 2015). 
However, these efforts are only useful for the content they have explicitly been developed for—the 
considerable work that goes into such efforts does not translate directly for other content areas. One 
possible solution would be to ask students to think about the kinds of errors that might be made and 
explain to themselves why those solutions would be incorrect. But are students able to anticipate the 
types of errors other students might make? And might explaining worked examples improve 
students’ ability to anticipate errors? These questions are a main focus of the present study. 


The Present Study 

There were two main purposes in this study. First, it aimed to replicate and extend previous 
results showing that the use of worked examples with self-explanation prompts is effective for 
students learning mathematics in real world classrooms. Prior results have found that in traditional 
mathematics classes, where procedural knowledge is typically emphasized and conceptual 
knowledge may be lacking, assignments containing correct and incorrect examples to explain along 
with problems to solve were more effective for improving both conceptual and procedural knowledge 
than assignments in which students solved all of the problems themselves. In the present study, 
correct, incorrect, and partially completed examples were incorporated into reform mathematics 
classrooms, where conceptual knowledge is typically emphasized through rich problem-based 
activities, but procedural knowledge may suffer (NRC, 2004). We hypothesize that worked examples 
will still be effective in a reform classroom and investigate differences in effectiveness for 
conceptual vs. procedural knowledge. 

The second purpose of this study was to examine students’ ability to anticipate the types of errors 
other students might make when solving equations, and determine if this ability is improved after 
experience explaining correct and incorrect examples, which highlight features of problems for 
which students might have misconceptions and demonstrate incorrect methods for solving the 
problems. We will examine the types of errors that students anticipate and determine how the 
anticipated types of errors change after instruction in general and/or the worked examples 
intervention. 


Methods 


Participants 

Seventy-five 8™ grade Algebra I students from an inner-ring suburban middle school in the 
Midwestern United States participated in the study (55% female; 59% African American, 21% 
White, 15% American Indian/Alaskan, 4% Asian, and 1% classified as other ethnicities). The study 
employed a quasi-experimental design where students were assigned to the treatment and control 
groups according to their rostered section of Algebra I. The four sections of Algebra I were taught 
by two teachers, with each teacher having one treatment and one control class. In all, 37 students 
(49%) participated in the experimental group while 38 students (51%) participated in the control 
group. All of the Algebra I classes utilized the Connected Mathematics Project 2 Curriculum (CMP2; 
Lappan, Fey, Fitzgerald, Friel, Phillips, 2006) CMP2 includes rich, problem-based investigations 
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during classroom lessons, and provides a variety of practice problems for students to solve afterward 
as classwork and/or homework. This study took place during the Say It With Symbols unit, which 
focuses on understanding symbols in algebraic equations. 


Measures and Coding 

A single, experimenter-designed paper-and-pencil test was utilized for this study and 
administered as both pretest and posttest. The test included three types of items: conceptual 
knowledge, procedural knowledge, and error anticipation. 

Conceptual knowledge. To examine students’ conceptual knowledge about problem features, 
we used 21 items focused on the meanings of different terms in an equation, identification of 
equivalent expressions, and categorization of functions as linear, quadratic, or exponential. The 
percentage of these items answered correctly was computed for each student at pretest and at 
posttest. 

Procedural knowledge. To examine procedural skill, we used 9 items which asked students to 
solve multi-step equations, simplify expressions using the distributive property, and evaluate 
formulas at given values. The percentage of these items answered correctly was computed for each 
student at pretest and at posttest. 

Error anticipation. To evaluate ability to identify errors that others might make when solving 
multi-step equations, we utilized one item which asked students what mistakes they thought a 
seventh-grader might make in solving the equation 5x — 2 = 8. In total, they were asked to identify 
two potential errors. Student responses were coded first in terms of whether or not the provided 
responses were reasonable, and then by the type of error referenced: mistakes involving variables 
(e.g., handle the coefficient separately from the variable), like terms (e.g., subtract 2 from 5x), 
negative signs (e.g., subtract 2 from both sides instead of adding 2), equals sign (e.g., perform an 
operation to one side and not the other), operations (e.g., adding two numbers instead of 
multiplying), other reasonable errors, or unreasonable errors. For each type of error, students were 
scored in terms of whether at least one of their responses fit in that category. 


Procedure 

Prior to beginning the Say It With Symbols unit, all students took the paper-and-pencil pretest. 
Instruction for the treatment and control classrooms was kept constant within teacher (e.g., Teacher 
A provided the same lesson to both her treatment class and her control class). The only difference 
between conditions is that when students were to work on their practice problems, treatment classes 
were given an adapted version of the Say It With Symbols book. In the adapted book, approximately 
26% of the practice problems were replaced with a correct, incorrect, or partial example of a solution 
to that problem. Teachers could assign as many or as few practice problems as they desired as long as 
the same items were assigned to both their treatment and control groups. When the unit was 
complete, teachers administered the paper-and-pencil posttest to all students. 


Results 

To examine the effectiveness of the treatment for improving students’ conceptual and procedural 
knowledge, we conducted a 2 (condition: treatment vs. control) x 2 (time: pretest vs. posttest) x 2 
(measure: conceptual vs. procedural) RMANOVA, with repeated measures on time and measure. The 
analysis yielded a main effect of time (F (1, 73) = 23.50, p < .0.001, n; = 0.24), with students 
performing better at posttest (M = 52%) than at pretest (M = 41%). The main effect of measure was 
also significant (F (1, 73) = 52.97, p < .0.001, 7; = 0.42), with students performing better on 
conceptual items (M = 52%) than on procedural items (M = 41%). There was a significant time by 
measure interaction, F' (1, 73) = 15.43, p< .0.001, n; = 0.17), with students improving more from 
pretest to posttest on procedural items (31% to 48%) than conceptual items (51% to 55%). Finally, 
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there was a significant interaction between time and condition, F (1, 73) = 4.66, p = .0.034, n5 = 
0.04, revealing that students in the treatment group improved more from pretest to posttest (42% to 
57%) than students in the control group (40% to 46%). No other main effects or interactions reached 
significance. 

To examine the types of errors anticipated by students and how this anticipation changes with 
instruction and worked examples intervention, we conducted a 2 (condition: treatment vs. control) x 
2 (time: pretest vs. posttest) x 6 (error type: variable, like terms, negative signs, equals signs, 
operations, and other reasonable errors) RMANOVA, with repeated measures on time and error type. 
The analysis yielded a main effect of time, F (1, 73) = 23.92, p < .0.001, n; = 0.25), with a greater 
likelihood of anticipating reasonable error responses at posttest (23%) than at pretest (14%). There 
was also a significant interaction between time and condition, F (1, 73) = 4.77, p = .0.032, n5 = 
0.06), with the likelihood of anticipating reasonable errors increasing more from pretest to posttest 
for the treatment group (12% to 24%) than for the control group (17% to 22%). The main effect of 
error type also reached significance, F (5, 69) = 10.51, p < .0.001, n;, = 0.43). Post-hoc tests with 
Bonferroni correction revealed that variable, like terms, and negative sign errors were more likely to 
be anticipated than equals sign and other reasonable errors; variable errors were also more commonly 
anticipated than operations errors (see Figure 1). Finally, there was a significant interaction between 
time and error type F (5, 69) = 2.47, p = .0.041, n; = 0.15). Follow-up paired-sample t-tests revealed 
that the likelihood of anticipating three types of errors increased from pretest to posttest: Like terms 
(12% to 39%; t(74) = 4.37, p < .001), operations (8% to 19%; t(74) = 2.04, p = .045), and other 
reasonable errors (4% to 15%; #(74) = 2.38, p = .020). 


Discussion 

Results from the present study replicate and extend prior studies on the effectiveness of worked 
examples in mathematics learning in two important ways. First, they demonstrate that incorporating 
worked examples into students’ practice in reform classrooms is a useful practice. Even though such 
classrooms are more focused on conceptual understanding than traditional classrooms, studying and 
explaining worked examples leads to improved learning over problem-solving practice alone. In 
particular, improvement in procedural knowledge as a result of worked examples may be especially 
helpful in reform classrooms. 

The second type of extension provided by the present study is that practice containing worked 
examples, many of which prompt students to reflect on incorrect procedures, leads to an increased 
likelihood that students will be able to anticipate errors that other students might make. If teachers 
want to make learning from errors a more prominent part of their classrooms but do not have access 
to—or time to create—relevant error-centered lessons, it may be desirable to have students that can 
think about potential errors on their own and reflect on why those anticipated errors are problematic. 
Introducing assignments with incorrect examples earlier in the process may train students to 
anticipate such errors on their own. However, further research is needed to examine the mechanism 
underlying this improvement and determine whether and how error anticipation skill transfers from 
one particular type of content to another. For example, if students’ error anticipation skill improves 
after worked example assignments in a linear equations unit, would they be more likely to anticipate 
errors for solving quadratic equations? For graphing linear functions? For geometry? The answers to 
these questions are likely different depending on whether error anticipation improves because 
students get used to reflecting on errors in general or because they are exposed to examples of errors 
specific to that content area. 
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Figure 1: Likelihood of anticipation by error type. 


The present study also revealed differences in the types of errors students tend to anticipate. 
Errors dealing with variables, like terms, and negative signs were the most frequently anticipated 
across time points, and anticipation of like terms, operations, and other reasonable errors were most 
likely to increase after students gained more knowledge about the content area. This is interesting for 
several reasons. For instance, prior research has identified the types of algebraic errors that are the 
most prevalent for Algebra I students (Booth, Barbieri, Eyer, & Pare-Blagoev, 2014). This work 
suggests that one of the types of highly-anticipated errors in the present study—those involving 
negative signs—are highly prevalent in equation-solving activities, but that the other two highly- 
anticipated error types—variables and terms errors—are not among the most prevalent in that content 
area. This indicates that students may be likely to anticipate negative sign errors because they see (or 
make) them frequently. However, it is not clear why students would be likely to anticipate variables 
or terms errors if they are not frequently made. Further study, perhaps with think aloud data 
collection, is necessary to determine how students come up with the errors they anticipate. A 
combination of think aloud and classroom observation data would also enable us to examine 
differences in these anticipations at pretest vs. posttest and determine why certain types of errors 
become easier to recognize after instruction on the topic. Knowing what the teachers are highlighting 
in their lessons may help explain why anticipation of certain error types increases while other types 
do not. If a teacher aims to get students to anticipate a wider variety of errors, further intervention 
targeted at helping students notice less anticipated errors may be necessary. 
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